Facial Expression Recognition using Neural Networks
Automatic analysis of facial expressions has attracted increasing attention from researchers in various fields such as psychology, computer science, linguistics, neuroscience and related fields. This work proposes a system for automatic recognition of facial expressions. Initially 20 facial landmarks are extracted from each face¹. Subsequently, the GPA (Generalized Procrustes analysis)² is applied to normalize all landmarks (i.e. translation, rotation, reflection and scaling). All landmarks are used to classify seven different facial expressions: neutral, angry, disgust, fear, happy, sad and surprise. The system was evaluated using MUG Facial Expression and FG-NET image databases. The experimental results with neural networks show 97.62% of accuracy for MUG database and 86.05% of accuracy for FG-NET database.
¹ The shape of each face is defined by a set of 20 facial landmarks. In http://bit.ly/Ufnake we explains how to extract the facial landmarks.
² Another good introduction to GPA: http://www.sensorysociety.org/ssp/wiki/Generalized_Procrustes_Analysis/
The MLP network was configured as follows:
a) the input layer has 40 neurons (4 landmarks x 2 cartesian coordinates x 5 facial regions).
b) the hidden layer were evaluated with 10 to 16 neurons.
c) the output layer has 7 neurons (one neuron for each expression: neutral, angry, disgust, fear, happy, sad and surprise).
Transfer function: sigmoid.
The training algorithm used in this work is the CGP (Conjugate gradient backpropagation with Polak-Ribiere updates) proposed by Polak and Ribiere (1969). The CGP has achieved better results than tradicional backpropagation algorithm.
Experimental Results
Database
MUG Facial Expression
Face and Gesture Recognition Research Network (FG-NET)
------------------------------------------------------------------------------------------------------------------------------------------------------
Evaluation using MUG Facial Expression database
Evaluation using FG-NET database
Sample from FG-NET database using Neural Network
------------------------------------------------------------------------------------------------------------------------------------------------------
* Emotion and Paralinguistic Communication
* Clinical Psychology
* Psychiatry
* Neurology
* Pain
* Assessmen
* Lie Detection
* Clinical Psychology
* Intelligent Environments
* Multimodal Human-Computer Interface (HCI)
-----------------------------------------------------------------------------------------------------------------------------------------------------
Link to Full Dissertation / MSc Thesis (in Brazilian Portuguese):
Presentation slides (in Brazilian Portuguese):
Publications
2014 - Silva, Caroline; Sobral, Andrews; Tavares Vieira, Raissa. "An automatic facial expression recognition system evaluated with different classifiers". X Workshop de Visão Computacional (WVC’2014), Uberlândia, Minas Gerais, Brazil, October, 2014. [PDF] [CODE]
You can find a video tutorial on how to use the code at: https://youtu.be/VQYG-O6ui2E
2014 - Silva, Caroline; Sobral, Andrews; Tavares Vieira, Raissa. "Facial expression recognition in static images by generalized procrustes analysis". X Workshop de Visão Computacional (WVC’2014), Uberlândia, Minas Gerais, Brazil, October, 2014. [PDF]
